# ReadMe file for "Measuring the Impact of Appointee Vacancies on U.S. Federal Agency Performance"

## Code

### Main Text

- `01_r_p_l_jop_012724_del.do`: Code for regression models in main text (excluding Table 2) and for Tables D1 and D2 in Appendix D, Table F3 in Appendix F, Appendix I, Appendix G, and Footnote 6
- `02_perf_figures_main_text.R` Code to create figures 1, 3, and 4 in the main text
- `03_fig_2_main_text.R`: Code to create figure 2 in the main text
- `04_table_2.R`: Code to estimate models in table 2 (data not included)

### Appendices

- `A1_pty_non_response.R`: Table A1 and analysis of partisan non-response bias in Appendix A (data not included)
- `B1_performance_ratings_comp.R`: Figure B1 and analysis in Appendix B
- `D1_validation_figure.R`: Figure D1 - see `01_r_p_l_jop_012724_del.do` for Tables D1 and D2
- `F_ovb_sensitivity.R`: Code for Tables F1, F2, F4, and F5 (data for Table F2 not included)
- `H1_pty_dis_figure.R`: Figure H1

### Estimates of latent traits

#### Performance

- `01_performance_ratings_get_data.R`: Format the survey data for model estimation
- `02_performance_ratings_estimation_stan.R`: Estimate Bayesian models of agency performance
- `03_performance_ratings_estimation_pid_models.R`: Estimate Bayesian models of agency performance accounting for partisanship
- `04_performance_model_diagnostics.html`: Document containing diagnostics of Bayesian models of agency performance
- `04_performance_model_diagnostics_pid.R`: Code for performing diagnostics
- `05_performance_model_diagnostics_pid.html`: Document containing diagnostics of Bayesian models of agency performance accounting for partisanship
- `05_performance_model_diagnostics_pid.R`: Code for performing diagnostics
- `06_performance_ratings_format.R`: Format the agency performance estimates for use and publication
- `07_performance_ratings_format_pid.R`: Format the partisan agency performance estimates for use and publication
- `08_performance_ratings_format_all.R`: Combine the nonpartisan and partisan estimates into one data file

#### Workforce Skills

- `01_skills_ratings_get_data_2020.R`: Format the 2020 survey data for model estimation
- `02_skills_ratings_estimation.R`: Use the responses from the 2014 and 2020 surveys to generate estimates of workforce skills.

#### Stan models

- `ratings_model_informed_sd.stan`: Stan model for nonhierarchical estimates with informed priors
- `ratings_model_informed_sd_hierarchical_multi.stan`: Stan model for hierarchical estimates with informed priors
- `ratings_model_informed_sd_hierarchical_multi_pid.stan`: Stan model for hierarchical estimates with informed priors that include indicator variables for appointees, Democrats, and Republicans
- `ratings_model_naive.stan`: Stan model for nonhierarchical estimates with naive priors
- `ratings_model_naive_hierarchical_multi.stan`: Stan model for hierarchical estimates with naive priors
- `ratings_model_naive_hierarchical_multi_pid_seperate.stan`: Stan model for hierarchical estimates with naive priors that allow Republicans and Democrats to have separate ratings with estimates for a subset of agencies constrained to be the same for all raters to bridge the partisan scales
- `ratings_model_naive_sd_hierarchical_multi_pid.stan`: Stan model for hierarchical estimates with naive priors that include indicator variables for appointees, Democrats, and Republicans

## Data 

### Data availability

Individual-level survey data are not included in these replication files in accordance with IRB protocols to protect respondent anonymity. We provide a list of analysis for which replication data are not available below. The code is nonetheless included to make as much material for replication available as possible.

### Data files

- `01_lpr_012824.xlsl`: Data for estimating models in main text and appendices - used by `01_r_p_l_jop_012724_del.do`
- `02_q_vacancies_data_trump.dta`: Data for estimating models in appendices - used by `01_r_p_l_jop_012724_del.do`
- `03_gao_recs_replication.dta`: Data for estimating models in appendices - used by `01_r_p_l_jop_012724_del.do`
- `07_performance_ratings_all_fmtd.RData`: All the estimates of agency performance
- `01_dis_data.csv`: Partisan disagreement estimates from replication files for Richardson (2024)

## Items for which data is not available

- Table 2 (main text)
- Table A1 and analysis of partisan non-response bias in Appendix A
- Table F2
- Generating estimates of latent performance and workforce skill

## Codebooks

- `Codebook for Agency Performance Data_012824`: Codebook for `01_lpr_012824.xlsl`.
- `Codebook for Agency Quarterly Vacancies Data`: Codebook for `02_q_vacancies_data_trump.dta`
- `Codebook for GAO Open Recommendations Data`: Codebook for `03_gao_recs_replication.dta`
- `Codebook for Agency Performance Ratings`: Codebook for `07_performance_ratings_all_fmtd.RData`

## Software

The Bayesian models estimating agency performance were fit with RStan version 2.21.3 and R 4.0.0. Necessary code is found in the `ratings_code/performance_ratings` folder and the corresponding `Stan` models are in `stan_models` folder. The Bayesian model for estimating the workforce skill was fit with `runjags` using JAGS 4.2.0 and R 4.0.0. The necessary code is in the `ratings_code/skills_ratings` folder. The JAGS model is on lines 362 to 411 of the `02_skills_ratings_estimation.R` script.

